Papers with natural language processing models
Entity Recognition at First Sight: Improving NER with Eye Movement Information (N19-1)
Copied to clipboard
| Challenge: | Previous studies have shown eye-tracking data can be used to improve natural language processing models. |
| Approach: | They leverage eye movement features from three corpora with recorded gaze information to augment a neural model for named entity recognition with gaze embeddings. |
| Outcome: | The proposed model outperforms baseline models on both individual datasets and in cross-domain settings. |
Bias and Fairness in Natural Language Processing (D19-2)
Copied to clipboard
| Challenge: | a tutorial will review the history of bias and fairness studies in machine learning and language processing . |
| Approach: | This tutorial reviews the history of bias and fairness studies in machine learning and language processing . it presents recent community effort to quantify and mitigat bias in natural language processing models . |
| Outcome: | This tutorial reviews the history of bias and fairness studies in machine learning and language processing . it aims to quantify and mitigate bias in natural language processing models for a wide spectrum of tasks . |
DOCMASTER: A Unified Platform for Annotation, Training, & Inference in Document Question-Answering (2024.naacl-demo)
Copied to clipboard
| Challenge: | DOCMASTER is a platform for annotating PDF documents, model training, and inference, tailored to document question-answering. |
| Approach: | They propose to integrate layout information into a unified platform for annotating PDF documents, model training, and inference tailored to document question-answering. |
| Outcome: | The proposed platform is designed for annotating PDF documents, model training, and inference, tailored to document question-answering. |
Robustness to Modification with Shared Words in Paraphrase Identification (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Paraphrase identification models have been shown to be vulnerable and lack robustness in tasks such as text classification and natural language inference. |
| Approach: | They propose to modify an example such that a target model makes a wrong prediction by using beam search constrained by heuristic rules and a BERT-masked language model to generate substitution words compatible with the context. |
| Outcome: | The proposed model performance drops dramatically on modified examples, revealing the robustness issue. |
A Simple Approach for Handling Out-of-Vocabulary Identifiers in Deep Learning for Source Code (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to handle out-of-vocabulary identifiers are not suitable for source code processing. |
| Approach: | They propose a method to handle out-of-vocabulary identifiers by identifies anonymization . they show that the method significantly improves the performance of the Transformer . |
| Outcome: | The proposed method significantly improves the performance of the Transformer in two code processing tasks. |
Parallax: Visualizing and Understanding the Semantics of Embedding Spaces via Algebraic Formulae (P19-3)
Copied to clipboard
| Challenge: | Embeddings are a fundamental component of many modern machine learning and natural language processing models. |
| Approach: | They propose a tool for visualizing embedding spaces using parametric projections . they demonstrate the power of Parallax and propose % task-oriented approach . |
| Outcome: | The proposed tool is based on two-dimensional projections without interpretable semantics . it enhances interpretability and allows for more fine-grained analysis . |
Neural Cross-Lingual Named Entity Recognition with Minimal Resources (D18-1)
Copied to clipboard
| Challenge: | Named-entity recognition (NER) models are highly dependent on large amounts of labeled data. |
| Approach: | They propose a method that finds translations based on bilingual word embeddings . they also propose 'self-attention' which allows for a degree of flexibility with respect to word order . |
| Outcome: | The proposed method achieves state-of-the-art or competitive performance on common languages with lower resource requirements than previous approaches. |
Efficient Meta Lifelong-Learning with Limited Memory (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing natural language learning models fail to continuously learn new tasks as they are re-trained throughout their lifetime. |
| Approach: | They propose a meta-lifelong framework that combines three common lifelong learning principles . they propose to store past examples in episodic memory and replay them at training and inference time . |
| Outcome: | The proposed framework achieves state-of-the-art performance using 1% memory size and narrows the gap with multi-task learning. |
A Unified Evaluation Framework for Novelty Detection and Accommodation in NLP with an Instantiation in Authorship Attribution (2023.findings-acl)
Copied to clipboard
| Challenge: | State-of-the-art natural language processing models have been shown to achieve remarkable performance in ‘closed-world’ settings where all the labels in the evaluation set are known at training time. |
| Approach: | They propose a multi-stage task to evaluate a system's performance on pipelined novelty ‘detection’ and ‘accommodation’ tasks. |
| Outcome: | The proposed model performs poorly in ‘closed-world’ settings where all the labels in the evaluation set are known at training time. |
Great Power, Great Responsibility: Recommendations for Reducing Energy for Training Language Models (2022.findings-naacl)
Copied to clipboard
| Challenge: | Recent studies show that the energy requirements of current NLP models are growing at a rapid, unsustainable pace. |
| Approach: | They investigate ways to measure energy usage and different hardware settings that can be tuned to reduce energy consumption for training and inference for language models. |
| Outcome: | The proposed techniques can reduce energy consumption for training and inference for language models. |
Is the Best Better? Bayesian Statistical Model Comparison for Natural Language Processing (2020.emnlp-main)
Copied to clipboard
| Challenge: | a recent study raises concerns about the use of standard splits to compare models . we compare the performance of six English part-of-speech taggers to those of other models based on standard split analysis . |
| Approach: | They propose a Bayesian statistical model comparison technique using k-fold cross-validation . they rank six English part-of-speech taggers across two data sets and three evaluation metrics . |
| Outcome: | The proposed method ranks English part-of-speech taggers on two data sets and three evaluation metrics. |
ODD: A Benchmark Dataset for the Natural Language Processing Based Opioid Related Aberrant Behavior Detection (2024.naacl-long)
Copied to clipboard
Sunjae Kwon, Xun Wang, Weisong Liu, Emily Druhl, Minhee Sung, Joel Reisman, Wenjun Li, Robert Kerns, William Becker, Hong Yu
| Challenge: | Opioid related aberrant behaviors (ORABs) present novel risk factors for opioid overdose. |
| Approach: | They propose to use a biomedical natural language processing benchmark dataset to classify ORABs from patients’ EHR notes into nine categories: confirmed aberrant behavior, suggested aberrant behaviors, Opioids, indication, diagnosed opioid dependency, Benzodiazepines, medication changes, and Central Nervous System-related. |
| Outcome: | The proposed dataset outperforms two state-of-the-art models in most categories and the gains are especially higher among uncommon classes. |
GradSim: Gradient-Based Language Grouping for Effective Multilingual Training (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies show that not all languages positively influence each other . multilingual training can help in those cases by sharing knowledge across languages . |
| Approach: | They propose a gradient similarity-based language grouping method for multilingual training that is better correlated with cross-lingual model performance. |
| Outcome: | The proposed method leads to the largest performance gains on a multilingual dataset and is better correlated with cross-lingual model performance. |
MindGames: Targeting Theory of Mind in Large Language Models with Dynamic Epistemic Modal Logic (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Theory of Mind (ToM) is a critical component of intelligence but its assessment remains the subject of heated debates. |
| Approach: | They propose to use dynamic epistemic logic to isolate a particular component of ToM and generate controlled problems in English natural language. |
| Outcome: | The proposed language model scales from 70M to 6B and 350M to 174B do not consistently yield better results than random chance. |
Attention Mechanism with Energy-Friendly Operations (2022.findings-acl)
Copied to clipboard
| Challenge: | Empirical results show that attention mechanism can be improved from the energy consumption aspects. |
| Approach: | They propose to replace multiplications with either selective operations or additions to reduce energy consumption. |
| Outcome: | The proposed model achieves competitable accuracy while saving 99% and 66% energy during alignment calculation and the whole attention procedure. |
Transferable Neural Projection Representations (N19-1)
Copied to clipboard
| Challenge: | Neural word embeddings require lookup and a large memory footprint making it hard to deploy on-device. |
| Approach: | They propose a skip-gram based architecture coupled with Locality-Sensitive Hashing projections to learn efficient dynamically computable representations. |
| Outcome: | The proposed model performs better than previous models on multiple NLP tasks. |
HarfoSokhan: A Comprehensive Parallel Dataset for Transitions between Persian Colloquial and Formal Variations (2026.eacl-long)
Copied to clipboard
Hamid Jahad Sarvestani, Vida Ramezanian, Saee Saadat, Neda Taghizadeh Serajeh, Maryam Sadat Razavi Taheri, Shohreh Kasaei, MohammadAmin Fazli, Ehsaneddin Asgari
| Challenge: | A wide array of NLP/NLU models have been developed for the Persian language but performance drops when applied to the colloquial form of Persian. |
| Approach: | They propose to use a large-scale colloquial to formal Persian parallel dataset to train a GPT2 model that exhibited remarkable proficiency in colloqual to informal text style transfer. |
| Outcome: | The proposed dataset outperforms OpenAI’s GPT-3.5-turbo model and a leading rule-based system in colloquial to formal Persian conversion. |
SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing (2022.acl-long)
Copied to clipboard
Junyi Ao, Rui Wang, Long Zhou, Chengyi Wang, Shuo Ren, Yu Wu, Shujie Liu, Tom Ko, Qing Li, Yu Zhang, Zhihua Wei, Yao Qian, Jinyu Li, Furu Wei
| Challenge: | Existing work shows that pre-trained models can improve in various natural language processing tasks. |
| Approach: | They propose a unified-modal encoder-decoder framework that pre-trains speech-text representations using large-scale unlabeled speech and text data. |
| Outcome: | The proposed framework is superior to existing models on speech-to-text processing tasks. |
OCR Post Correction for Endangered Language Texts (2020.emnlp-main)
Copied to clipboard
| Challenge: | Currently, there is little to no data available to build natural language processing models for endangered languages. |
| Approach: | They propose a benchmark dataset of transcriptions for scanned books in three critically endangered languages and a method to improve OCR in these data-scarce settings. |
| Outcome: | The proposed method reduces the recognition error rate by 34% across the three endangered languages. |
Error Analysis of NLP Models and Non-Native Speakers of English Identifying Sarcasm in Reddit Comments (2024.lrec-main)
Copied to clipboard
| Challenge: | sarcasm detection remains an issue for both humans and natural language processing models . |
| Approach: | They analysed 300 comments from the FigLang 2020 Reddit Dataset and 39 non-native speakers of English to see if they were sarcastic. |
| Outcome: | The results show that the models and models have similar performance and weaknesses when the comments include political topics or are phrased as questions. |
HateGAN: Adversarial Generative-Based Data Augmentation for Hate Speech Detection (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods to detect online hate speech depend heavily on labeled datasets for training, which results in poor detection performance of the hate speech class. |
| Approach: | They propose a deep generative reinforcement learning model which augments two commonly-used hate speech detection datasets with the HateGAN generated tweets. |
| Outcome: | The proposed model improves the detection performance of hate speech class regardless of the classifiers and datasets used in the detection task. |
A Rewriting Approach for Gender Inclusivity in Portuguese (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent research has focused on gender-inclusive and gender-neutral language . however, current approaches to gender- neutral rewriting for gendered languages rely on large datasets . |
| Approach: | They propose a rule-based and a neural-based tool for gender-neutral rewriting for Portuguese, a heavily gendered Romance language. |
| Outcome: | The proposed model fine-tunes large multilingual machine translation models on examples generated by the rule-based model. |
Bayelemabaga: Creating Resources for Bambara NLP (2025.naacl-long)
Copied to clipboard
| Challenge: | a lack of well-structured multilingual datasets remains a challenge for machine translation in under-resource languages. |
| Approach: | They propose to create a multilingual dataset for machine translation in the Bambara language, the vehicular language of Mali. |
| Outcome: | The proposed dataset is the most extensive curated multilingual dataset for machine translation in the Bambara language, the vehicular language of Mali. |
Split and Rephrase with Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Split and Rephrase (SPRP) tasks require modelling complex grammatical aspects to provide optimal splits and appropriate rephrasing. |
| Approach: | They evaluate large language models on the Split and Rephrase task . they show they can provide large improvements over the state of the art on main metrics . |
| Outcome: | The proposed model outperforms the state-of-the-art model on the Split and Rephrase task on the main metric, but still lacks in splitting compliance. |
Estimating predictive uncertainty for rumour verification models (2020.acl-main)
Copied to clipboard
| Challenge: | Inability to correctly resolve rumours can have harmful real-world consequences. |
| Approach: | They propose a method for incorporating model and data uncertainty estimates into natural language processing models for automatic rumour verification. |
| Outcome: | The proposed methods filter out erroneous model predictions and prioritise them for a human fact-checker. |
Functionality learning through specification instructions (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Creating or annotating instances targeting specific functionalities is costly and further training models is expensive. |
| Approach: | They propose to use specification instructions to create specification-augmented prompts for each functionality in a suite and combine them with language models pre-trained on natural instruction data. |
| Outcome: | The proposed test suites can assess models’ performance on specific functionalities on four tasks and models of diverse sizes and families. |
Contrasting Human- and Machine-Generated Word-Level Adversarial Examples for Text Classification (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has raised the question of whether valid adversarial inputs are feasible. |
| Approach: | They analyze how human-generated adversarial examples compare to the best algorithms . they use crowdsourcing to modify words in an input text with immediate feedback . |
| Outcome: | The proposed algorithms are not more efficient than the best to generate natural-reading, sentiment-preserving examples. |
Mathematical Entities: Corpora and Benchmarks (2024.lrec-main)
Copied to clipboard
| Challenge: | a limited amount of annotated data is available for mathematical language processing . mathematics is a highly specialized domain with its own unique set of challenges . |
| Approach: | They provide annotated corpora that can be used to study the language of mathematics . they provide part-of-speech tags, lemmas, and dependency trees . |
| Outcome: | The proposed corpora provide part-of-speech tags, lemmas, and dependency trees . the learning assistant grants access to the content of the corporata in a context-sensitive manner . |
Adversarial Text Generation by Search and Learning (2023.findings-emnlp)
Copied to clipboard
Guoyi Li, Bingkang Shi, Zongzhen Liu, Dehan Kong, Yulei Wu, Xiaodan Zhang, Longtao Huang, Honglei Lyu
| Challenge: | Existing text generation methods only use heuristic replacement strategies or language models to generate replacement words at the word level. |
| Approach: | They propose a search and learning framework for Adversarial Text Generation by Search and Learning to evaluate the robustness of natural language processing models. |
| Outcome: | The proposed methods are significantly superior to the most advanced methods in terms of attack efficiency and adversarial text quality. |